1. Field of the Invention
The invention relates to a method of computing a hologram. In particular, it relates to a method of generating holograms using electro-holography. Electro-holography aims at realising computer-generated holograms in real-time (i.e. a reconstructed object can be generated from encoded holographic data in a short space of time). A holographic display typically contains an array of controllable pixels; the pixels reconstruct object points by electronically affecting the amplitude and/or phase of an illuminating light. Such an array is a form of spatial light modulator (SLM). The display may not be array based, but instead be continuous. For example, it may be a continuous SLM, including a continuous SLM with matrix control or an AOM (acousto-optic modulator).
A suitable display device to reconstruct video holograms by spatial amplitude modulation of a light pattern is, for example, a Liquid Crystal Display (LCD). However, this invention can also be applied to other controllable devices, which use coherent light for modulating a light wave front.
2. Definition of Terms and Background Concepts
In this document, the term ‘pixel’ denotes a controllable hologram pixel in the SLM; a pixel is separately addressed and controlled by a discrete value of a hologram point. Each pixel represents one hologram point of the video hologram. Hence, for a LCD, we use the term pixel to correspond to the individually addressable screen pixels. For a DLP, we use the term pixel to correspond to an individual micro-mirror, or a small group of micro-mirrors. On a continuous SLM, a pixel is a transient region on the SLM that represents one complex hologram point. The term pixel therefore means, at its most general, the smallest unit that can represent (e.g. display) one complex hologram point. To achieve colour encoding, each pixel may comprise sub-pixels for representing or displaying the colour hologram points in each of the three primary colours. Depending on the kind of video hologram encoding, further sub-pixels may be used for encoding or representing the primary colours of each colour hologram point. For instance, if Burckhardt encoding is used for a colour hologram, each pixel needs an arrangement of nine sub-pixels. For better clarity in this document each pixel is encoded by only one discrete hologram point value, containing an amplitude component and a phase component; said components may be zero. A dedicated controller or driver controls the sub-pixels using separate control signals for each sub-pixel. The controller or driver and the provision of control signals is not however the subject of this invention.
The term ‘pitch’ describes in this document the distance between the centres of two adjacent pixels of a SLM. It thus characterizes the display resolution.
An ‘observer window’ is a limited virtual zone through which the observer can see the entire reconstructed 3D scene with sufficiently high visibility. The observer window is located at or near the eyes of the observer. The observer window can be moved in the X, Y, and Z directions. Within the observer window, the wave fields interfere in a way that leads to the reconstructed object being visible to the observer. In one implementation of this invention, the scene is viewable through the observer window and is reconstructed inside a frustum that stretches between the edges of the observer window and the SLM. It is possible to include two observer windows, one for each eye. More sophisticated observer window arrangements are also possible. It is also possible to encode video holograms containing objects or entire scenes that the observer can see behind the SLM.
The term ‘encoding’ describes the way in which the SLM is supplied with control signals so that coherent light passing through the SLM or that is reflected by the SLM, reconstructs a three-dimensional scene.
A ‘light source’ according to this document is considered sufficiently coherent if the light is spatially coherent to an extent allowing interference, so that it allows holographic reconstruction with an adequate resolution in at least one dimension. Spatial coherence relates to the lateral extent of the light source. Conventional light sources, like LEDs or Cold Cathode Fluorescent Lamps, can also meet these requirements if they radiate light through an adequately narrow aperture. Light from a laser source can be regarded as emanating from a point source within diffraction limits. It leads to a sharp reconstruction of the object, i.e. each object point is reconstructed as a point within diffraction limits.
Light from a spatially incoherent source is laterally extended and causes a blurring or smearing of the reconstructed object. The degree of blurring or smearing is given by the broadened size of an object point reconstructed at a given position. In order to use a spatially incoherent source for hologram construction, a trade-off has to be found between reconstruction quality and brightness by adjusting the aperture width. A smaller aperture results in improved spatial coherence and hence lowers the degree of blurring or smearing. But a smaller aperture results in a lower brightness. The term ‘partial spatial coherence’ is used to describe such a light source.
Temporal coherence relates to the spectral line width of the light source. In order to ensure temporal coherence, the light must have an adequately narrow wavelength range. The spectral bandwidth of high-brightness LEDs is sufficiently narrow to ensure temporal coherence for holographic reconstruction. The diffraction angle at the SLM is proportional to the wavelength, which means that only a monochromatic source will lead to a sharp reconstruction of an object point. A broadened spectrum will lead to broadened object points and smeared or blurred object reconstructions. The spectrum of a laser source can be regarded as monochromatic. The spectral line width of a LED is sufficiently narrow to facilitate good reconstructions.
In most holographic systems, the encoded hologram is the transform of the 3D scene to be reconstructed. The term ‘transform’ should be expansively construed to include any mathematical or computational technique that is equivalent to or approximates to a transform. Transforms in the mathematical sense are merely approximations to physical processes more accurately described by Maxwellian wave propagation equations. Transforms such as Fresnel transforms (or the special class of transforms known as Fourier transforms) are second order approximations, but have advantages; because they are basically algebraic as opposed to differential, they can be handled in a computationally efficient manner, also, they can be accurately implemented in optical systems.
3. Description of Related Art
A drawback of 3D-autostereoscopic displays using conventional optics is a mismatch between parallax information and accommodation of the lens of the eye. On the one hand, the observer's eyes see different perspective views of a 3D-scene, which simulate a depth impression of objects at arbitrary distance. On the other hand, each perspective view is located on the display surface itself. Hence, the eye focuses on the display surface, and each eye sees a flat image. That causes a mismatch between seeing objects at arbitrary depth achieved by parallax information and the accommodation of the eyes to a fixed display surface. The mismatch may cause an unpleasant feeling and eye fatigue.
Known electro-holographic displays, for instance as described in document WO 01/95016, use a hologram matrix supplied with a pixel pattern of controllable openings which reconstructs objects of a 3D scene at correct depths. This can avoid the inconvenience of conventional stereoscopic displays. The diffraction from the small openings is used to reconstruct the 3D-scene. The wave fronts emerging from the openings converge in object points of the scene before they reach the observer. The smaller the diameter of openings of this hologram matrix, and thus the pitch, the larger is the diffraction angle. That causes a wide viewing angle for the use by the observer. Consequently, enlarging the viewing angle requires improved resolution.
The document from N. Fukaya, K. Maeno, K. Sato and T. Honda; “Eye-position tracking type electro-holographic display using liquid crystal devices”, S36-5, Post-Deadline Paper Asia Display '95 describes a method to expand the viewing zone in an electro-holographic display by eye position tracking. The document suggests that it is not necessary to project light from a holographic array into the whole area where the observer might be located. Rather, it is sufficient to restrict the illuminated area to the observer's eyes. Therefore, a large hologram array is divided into small pieces of separate holograms, each encoded with a pair of small holograms, instead of a single large hologram. That causes an observer to see the same 3-D object as if from one large hologram; each reconstructs the object and its viewing zone corresponds with each eye position. If the observer moves to another position, the observer gets the reconstruction and viewing zone from another pair of small holograms. This restriction facilitates the use of SLMs with significantly lower pixel count.
For tracking the observer's lateral (X,Y) movement, a controllable scanning mirror projects the light of the SLM to the observer's eyes. The tracking of observer's longitudinal (Z) movement occurs by changing the relative space between the small LCDs.
The document mentions a width of the reconstruction of 50 mm, which results in a relatively small angle in which the 3D-scene is rendered.
A disadvantage of this method is that manufacturing a holographic array containing multiple separate small LCDs is very difficult. Further, it has to be avoided that multiple reconstructions of the same object point of a 3D-scene can be seen. That limits the size of the SLM and hence the size of the object.
In order to reduce the enormous number of computations the patent specification WO 01/95016 A1 discloses a hologram calculation of only such parts of an electro hologram that can be seen directly by an observer or such parts that change. The electro hologram array consists of addressable sub-regions. The calculation is founded on a so-called effective exit pupil, which can coincide with the eye pupil of the observer in a specific position. If the observer position changes, a tracking device re-calculates the hologram part that generates the image for the new observer position.
However, this partly nullifies the reduction in the number of computations and the described solution does not avoid the disadvantage to need a large controllable SLM with extremely small pitch.
A device described in Document WO 2003/021363 (A1) for reconstructing computer-generated holograms decreases the requirements on the SLM by restricting the reconstruction to a horizontal-parallax only (HPO) hologram.
The illumination means is a line light source which generates monochromatic light with a bandwidth of less than 10 nm and which is coherent in the horizontal direction but incoherent in the vertical direction. The holographic reconstruction takes place only in the horizontal direction, whereas there is no holographic reconstruction in the vertical direction. This results in a reconstructed object with horizontal motion parallax. The perspective view does not change upon vertical motion. A HPO hologram requires less resolution of the SLM in the vertical direction than a full-parallax hologram. There is only a periodicity in the reconstruction direction, i.e. horizontally. The computational load is decreased for one-dimensional line holograms.
The document U.S. Pat. No. 6,927,886 (Plesniak) relates to computed holographic stereograms having a reconfigurable image surface that is spatially distinct from the hologram surface on which the holographic stereogram is encoded. A three-dimensional object or scene is captured or synthesized as a stack of one-dimensional holographic views (HPO-holograms) reconstructed by an array containing so-called holopixels having a structure different from a known pixel structure. Hardware generates computed diffractive patterns to produce viewable images and a generating module reconstructs the holographic stereograms by interference patterns at one or more image surfaces that are spatially distinct from the hologram surface.
The device projects one or more series of parallax views of a three-dimensional scene through one or more holographically reconstructed image planes. Via software, the image plane can be specified at any location off of the hologram plane and populated by a variable number of projector-pixels. Further, in a specific embodiment, the hologram surface and the image surface are separated by an adjustable distance. The image surface may be a variable depth and/or resolution.
In contrast to the pixels of the above-mentioned SLMs the holopixels have a very complicated structure and can reconstruct several holographic views.
Due to a reduction in the observer window to an extension that is just slightly larger than the pupil of an eye, applicant's former patent application WO 2004/044659 reduces significantly the requirements on the pitch of the SLM and the computational load of the holographic array. The device contains at least one light source, which provides sufficiently coherent light, a Fourier-transform lens, and a holographic array with a matrix of pixels that each contain one or more openings. The phase or amplitude of each opening is controllable and an observer plane is located in the image plane of the light source. In the observer plane, at least one observer window is formed in a periodicity interval as a transform of the video hologram, the observer window allowing an observer to observe a reconstruction of a three-dimensional scene. The maximal extent (i.e. X, Y dimensions) of the observer window may correspond to the periodicity interval in the Fourier transformation plane (which is equivalent to the image plane of the light source). A reconstruction frustum stretches between the display area and the observer window, said frustum containing the entire three-dimensional scene of the video hologram. As noted above, the observer window is limited to and positioned in relation to observer's eyes. Appendix II lists further aspects of and enhancements of WO 2004/044659; the enhancements are within the scope of the present invention.
4. Technical Background of the Invention
Common holographic arrays reconstruct a light wavefront of 3D-objects or 3D-scenes by coherent superposition of light waves. For that purpose, a spatial light modulator (SLM) displays a wave pattern encoded on the SLM (which may be a holographic array). The encoded hologram is the transform of the 3D scene. The SLM diffracts the light waves provided by a backlight and reconstructs the scene.
Fundamentally, the displaying of electro holograms, in which the holograms are sampled in hologram points, leads to a problem. Sampled holograms always have the property of periodic repetitions of the encoded wave pattern in periodicity intervals in the observer plane. These repetitions will cause multiple reconstructions of the same object or object points.
If the dimension of the reconstruction of the hologram exceeds the periodicity interval, adjacent diffraction orders will overlap. As the resolution is gradually decreased, i.e. as the pitch rises, the edges of the reconstruction will be distorted increasingly by overlapping adjacent diffraction orders. The usable extent of the reconstruction is thus gradually limited, because an overlapping of periodical reconstructed observer windows has to be avoided.
The viewing zone of a SLM depends on its maximum diffraction angle. The maximum is defined by the pixel pitch of the SLM.
As is generally known, in Fourier holograms the scene is reconstructed in a reconstruction plane as a direct or inverse Fourier transform of the encoding of the pixels of the holographic array (i.e. object reconstruction is at the Fourier plane of the array). This reconstruction is continued periodically at a periodicity interval, the extent of said periodicity interval being inversely proportional to the pixel pitch in the holographic array.
If larger periodicity intervals and thus greater viewing angles are to be achieved, the required pitch (and so the extent of sub-pixels of each pixel in the holographic array) comes closer to the wavelength of the illuminating light. The array area must be sufficiently large in order to be able to reconstruct large scenes. These two conditions (small pitch and large area) require a large holographic array having a great number of pixels.
For rendering the reconstructions of electro holograms, a sufficiently large viewing zone must be provided. In conventional holographic arrays, the viewing zone has to cover at least the eye separation, which requires a pixel pitch of ca. 10 μm at most. Costly hardware and high computational speeds are needed to calculate the electro-hologram in real-time.
The computational load on equipment that generates holograms in real time depends on the complexity of the holograms. A full-parallax hologram reconstructs an object holographically by coherent superposition of waves in the horizontal and vertical directions. Given a sufficiently large observer window or observer region, the reconstructed object can be seen with motion parallax in the horizontal and vertical directions, like a real object. However, a large observer region requires a high resolution SLM in both horizontal and vertical directions.
The requirements on the SLM and the computational device (e.g. dedicated ASIC, main device CPU, separate stand-alone device etc.) can be reduced by restriction to a horizontal-parallax only (HPO) hologram or a vertical-parallax only (VPO) hologram.
If a horizontal-parallax only hologram is used, the holographic reconstruction takes place only in the horizontal direction and there is no holographic reconstruction in the vertical direction. This results in a reconstructed object with horizontal motion parallax. The perspective view does not change upon vertical motion. A HPO hologram requires less resolution of the SLM in vertical direction than a full-parallax hologram. There is only a periodicity in the reconstruction direction, i.e. horizontally. The computational load is therefore decreased for one-dimensional line holograms.
A vertical-parallax only hologram where the reconstruction takes place only in the vertical direction is also possible but uncommon. This results in a reconstructed object with vertical motion parallax. There is no motion parallax in the horizontal direction. The different perspective views for the left eye and right eye have to be created separately. This can be done by temporal or spatial multiplexing of the observer windows.
Both VPO holograms and HPO holograms accomplish eye focussing (i.e. adapting the eye lens curvature) at the object distance.
It is common practice that the observer window of conventional electro-holographic displays is much larger than the pupil of the eye (i.e. that the reconstructed object can be seen correctly over a large area). A consequence is that much effort is needed to project light into regions of the space where no observer is located. Therefore, the performance required for the whole electro-holographic displays to control the optical wave front is extremely high.
Given a sufficiently large observer window or observer region, the reconstructed object facilitates motion parallax in horizontal and vertical direction, like a real object. However, a large observer region requires high resolution in both horizontal and vertical direction of the holographic array.
One known method to encode a hologram is by using a conventional liquid crystal display that modulates the amplitude by known Burckhardt encoding which is based on the detour-phase effect. The encoding needs three neighbouring sub-pixels per pixel and primary colour. This encoding provides three categories of diffraction orders called −1st, 0th, 1st, 2nd, 3rd, etc. diffraction order. The first category, the 0th, 3rd, etc. diffraction order contain un-diffracted light. These orders do not provide any reconstruction. The second category, the 1st, 4th, etc. diffraction order contains the reconstruction of the encoded object. In contrast, the third category, the −1st, 2nd, etc. diffraction order contains the reconstruction of a depth-inverted object. That means this reconstruction is incorrect. A correct reconstruction contains the 1st, 4th, etc. diffraction orders only. Due to the finite aperture of the LCD openings, the intensity of the diffraction pattern falls off towards higher diffraction orders. Therefore, it is advantageous to locate the observer window in the 1st diffraction order.
A periodicity interval provided by Burckhardt encoding comprises a group of three adjacent diffraction orders, like the 1st, 0th and −1st diffraction order. The size of each periodicity interval is given by Pdiffr=λ*d/p, where λ defines the wavelength of the illumination light; d is the distance between hologram and observer plane, p is the sub-pixel pitch.
As the object is reconstructed correctly in the 1st diffraction order only, the observer window can cover ⅓ of the periodicity interval Pdiffr only. Because the size of the periodicity interval depends on the wavelength of the illumination light too, for colour holograms the size of the observer window is limited by the shortest wavelength of the primary colour that is used.
If a phase-modulating SLM is used in a Fourier hologram, the periodicity interval contains no depth-inverted reconstruction of an object. However, there is also un-diffracted light. Therefore, not the whole periodicity interval can be used as an observer window. The un-diffracted light has to be excluded from the observer window also.
If a complex-modulating SLM is used, each single pixel can be used to encode one complex value. Therefore, each periodicity interval in the observer plane contains only one diffraction order. Hence, the whole periodicity interval can be used for an observer window.
In general, the observer window has to be located within one periodicity interval; however, depending on the encoding method of the complex-valued hologram on the SLM the observer window has to be smaller than a periodicity interval.
The propagation of light caused by an electro-hologram can be described by Fresnel transforms or Fourier transforms. Fresnel transforms describe the near-field light distribution whereas Fourier transforms describe the far-field light distribution at infinite distance. The far-field light distribution can be shifted to a finite distance by a focussing lens.
The solution known from patent application WO 2004/044659 is based on the idea to limit this encoded area such that light emanating from reconstructed scene points is confined to one observer window. Therefore, the device reconstructs a video hologram in one periodicity interval of the Fourier transform in an observer plane. The reconstructed three-dimensional scene can be observed through an observer window located in front of each eye. The reconstructed scene is visible inside the reconstruction frustum; the scene can thereby be reconstructed on, in front of or behind the array surface. This allows the use of a conventional array with resolution near 3 million pixels at reasonable hardware expenses and computing power.